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ABSTRACT 


Human computer interaction (HCI) for completely locked-in patients is a very 
difficult task. Nowadays, information technology (IT) is becoming an 
essential part of human life. Patients with completely locked-in state are 
generally unable to facilitate themselves by these useful technological 
advancements. Hence, they cannot use modern IT gadgets and applications 
throughout the lifespan after disability. Advancements in brain computer 
interface (BCI) enable operating IT devices using brain signals specifically 
when a person is unable to interact with the devices in conventional manner 
due to cognitive motor disability. However, existing state-of-the-art 
application specific BCI devices are comparatively too expensive. This paper 
presents a research and development work that aims to design and develop a 
low-cost general purpose HCI system that can be used to operate computers 
and a general purpose control panel through brain signals. The system is based 
on steady state visual evoked potentials (SSVEP). In proposed system, these 


electrical signals are obtained in response of a number of different flickering 
lights of different frequencies through electroencephalogram (EEG) electrodes 
and an open source BCI hardware. Successful trails conducted on healthy 
participants suggest that severely paralyzed subjects can operate a computer 
or control panel as an alternative to conventional HCI device. 
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1. INTRODUCTION 

Volume of stroke patients is quite high. As per a recent finding by ‘global burden of disease (GBD) 
2016 Stoke Collaborators’ [1], around 116.4 million stroke survivors’ cases in a year were reported 
worldwide who are living with disabilities. Donkor [2] indicated that severe brain injury results in up to 50% 
of survivors being chronically disabled. Guan described [3] that around 30% of stroke survivors require 
various forms of rehabilitation therapy especially for upper limb problems and loss of hand movements. 
Moreover, Jhonson et al. [4] presented that globally, 70% of strokes occur in low and middle-income 
countries. In the last four decades, the stroke incidence in low and middle-income countries has increased by 
more than 100% whereas during this period, stroke occurrence has declined by 42% in high-income 
countries. 

These figures indicate that a large number of human populations who are facing disabilities due to 
stroke belongs to low and middle-income countries. Due to severe motor disability, they are unable to use 
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computer and other IT devices through conventional means. An extensive research and development exercise 
is essentially needed to support the stroke victims in improving their quality of life and to enable them to 
continue using computers and IT devices. Advancements in brain computer interface (BCI) technology 
enabled the researchers to carryout extensive research in design and development of such types of low-cost 
human computer interface (HCI) systems that can be interfaced with the computers and other IT gadgets as a 
plug and play HCI device operated through brain waves. Hence, these HCIs can be used by the completely 
locked-in patients who cannot use keyboard, mouse and touch screen. 

The targeted HCI system exploits a neural process of steady state visual evoked potential (SSVEP). 
visual evoked potential (VEP) is the potential-difference on human scalp obtained from the EEG electrodes 
placed on the surface at visual cortex of the brain during a visual stimulus of flickering light. According to 
Wu et al. [5], “SSVEP system selects target by eye focus. SSVEP invoke when the retina is excited by a 
visual stimulus ranging from 3.5 Hz to 70 Hz, the brain generates electrical activity at the same frequency of 
visual stimulus or some harmonics”. Researchers are targeting SSVEP based BCIs in many areas. Hwang et 
al. have proposed a SSVEP based psychosomatic application [6] for spelling tasks by applying a Q-W-E-R- 
T-Y style layout keyboard based on 30 light emitting diodes (LEDs). 

In an interesting work [7], Volosyak et al. proposed a SSVEP-BCI that were assessed with a number 
of stimulation frequencies in an assignment related to spelling and concluded that fastest SSVEP response 
was achieved for the stimulus frequency of 6.67 Hz. Another similar work for spelling assignment by Cecotti 
suggested a SSVEP based self-paced BCI speller [8] which can work without any training from the user or 
from the signal processing section. Kim et al. [9] proposed a valuable point-and-click intra-cortical neural 
interface system (NIS) that permits tetraplegia subjects to operate a 2-dimensional computer cursor. The 
subjects can move the cursor on computer screen in any required direction, with hold and click on the area on 
screen through BCI. 

Liu et al. presented a SSVEP-BCI based augmented reality system based on HoloLens [10] and 
achieved significant accuracy and information transfer rate (ITR). Khatri and Farooq proposed a research 
work [11] that describes the use of SSVEP to develop a sentence speller to aid locked-in syndrome (LIS) 
patients. In an interesting work, Angrisani et al. [12] utilizes the architecture of typical SSVEP-BCI and 
proposed and tested a low-cost single channel SSVEP based instrument that can be used in control 
applications. Lin et al. developed a method [13] to operate a web browser using SSVEP combined with eye 
tracking through a 64-channel device. Similarly, Saravanakumar and Reddy [14] demonstrated that use of 
electro-oculogram (EoG) along with SSVEP can improve the performance. Bianchi et al. described a work 
[15] that studies the concept of providing analog like control using SSVEP through seven flickering stimuli. 

Ajami et al. [16] also worked on single channel SSVEP speller application aimed with user comfort. 
Hasan et al. described a direct SSVEP BCI System [17] that targeting higher information transfer rate to 
select commands in graphical user interface. Cao et al. recently presented a SSVEP BCI based Speller [18] 
for numerical inputs based on sliding control. Chiuzbaian et al. [19] discussed an innovative SSVEP based 
system to control a drone. Phothisonothai and Tantisatirapong [20] presented a SSVEP based virtual Thai 
keyboard for spelling and control tasks. Ramirez-Quintana et al. described their work [21] for SSVEP 
embedded processing with higher accuracy on signal channel input. Lin et al. also presented a recent work 
[22] on SSVEP based spelling system combined with eye tracking. Adams et al. [23] presented the concept 
of a smart home that can be controlled by SSVEP-BCI. Similarly, Park et al. presented a detailed work [24] 
on development of augmented reality and SSVEP-BCI based system to control home appliances. Garro et al. 
presented a SSVEP-BCI based alternative communication system [25] for the people with speech 
impairments. The concept of using EEG to perform robot motion control was proposed by Dewangga et al. 
[26]. The EEG data acquisition system can also be used to detect learning disorder in children using brain 
waves [27] as described by Kamaruddin et al. 

The usability and effectiveness of SSVEP-BCI based systems for common people are mainly 
depended on less number of electrodes to allow wear-ability and home use, low cost implementation through 
simple hardware and light weight algorithms and minimum training requirements to allow subject 
independency with less inter-subject variability. Joseph and Govindaraju have verified [28] that even a major 
reduction of EEG electrodes from 118 to 10 does not significantly affecting the BCI performance. Along 
with these features maintained with low-cost, more controllable options, and commands that can be triggered 
by the completely locked-in patient is a very important requirement. Inspired by abovementioned 
contributions that are based on expensive hardware, we have proposed a low-cost general purpose SSVEP- 
BCI based HCI system that can be used by locked-in patients in place of conventional HCI to operate 
computers and other devices through eight different controllable classes. 

The effectiveness of proposed system was tested through experiments on eight healthy subjects. The 
trials were conducted in accordance with concerned declaration. Experimental results demonstrated that the 
proposed low-cost system is providing sufficient interface with the computer and can be used by the targeted 
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individuals. The rest of the paper is organized is being as: section 2 presents materials and methods followed 
by experimental results and discussion in section 3. Finally, the conclusion and future recommendations are 
provided in section 4. 


2. RESEARCH METHODS 

This section discusses the research method on the material and techniques used in the testing of 
proposed system including system design, experimental setup, signal acquisition, processing, and control. 
The proof-of-principle was demonstrated by implementing the proposed HCI using a notebook computer and 
open-source low cost hardware module with an open-source software. 


2.1. System design 

The system comprises of four modules: a set of three EEG electrodes, a low-cost open source bio- 
signal amplifier, a notebook computer, and an output module as mentioned in Figure 1. Eight Ag/AgCl 
electrodes are placed on visual cortex at the back of scalp, one on forehead as ground and one on ear lobe as 
reference electrode. The low-cost open source amplifier (OpenBCI, U.S.A.) is a small and light-weight bio 
signal amplifier which is used to send SSVEP signals to the computer via bluetooth. SSVEP-EEG signals 
were sampled at 250 Hz. A portable battery pack of four AA size batteries is used to power the amplifier. The 
amplifier along with battery pack is completely head-mounted. 

The computer receives SSVEP signals from the amplifier on universal serial bus (USB) port through 
bluetooth dongle. After processing the signals through modified open source program, command signals 
elicited from SSVEP are sent to the output module. A customized open source code of OpenBCI software 
runs on the computer. The software is customized to provide user interface for output mode selection and to 
operate the output module through visualizing flickering LEDs. The raw EEG signals are also displayed. The 
output module is selectable by the user either to operate a control panel consists of eight discrete outputs or to 
work as SSVEP-BCI based computer mouse connected to the computer via USB cable. 


Data Acquisition Module Display Panel 


BioSignal Amplifier Unit 


Data Processing Module Control Module Control Panel 





Figure 1. Proposed system design 


2.2. Experimental setup and signal acquisition 

Eight healthy participants of normal or adjusted to normal eye-vision who were naïve to SSVEP- 
BCI were selected for trials through convenience sampling method. The participants were taken normal diet 
without consumption of alcohol or caffeine to avoid possible effect of SSVEP elicitation as described by 
Pradhan et al. [29]. Procedures and design of trials were in accordance with the principles of the helsinki 
declaration and experiment protocol was properly described to each participant. The trials were divided in 3 
stages in which participants are required to confirm the stimuli detection, to the operate a control panel of 8 
discrete outputs and to move the cursor on computer screen respectively through visualizing flickering LEDs. 

In each trial, participants were asked to sit on a chair in front of the computer screen at a distance of 
around 4 foots to get visual stimuli as shown in Figure 2. Flickering LEDs of 8, 10, 12, 14, 17, 19, 21, 23, 25, 
27, 29, 31, 33, 35, 37, and 40 Hz were used to get the stimuli. All the LEDs are of green color to exploit the 
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capability of maximum visual stimuli with highest level of comfort [30], [31]. Electrodes were placed at 
visual cortex around O1, O2, and Oz positions of 10-20 system of EEG electrode placement whereas reference 
and ground electrodes were placed at Fp2 and A2 respectively. 


BCI-Rehab Project 


Hamdard University 





Figure 2. Visual stimuli and corresponding amplitude 


The electrodes were connected to the head-mounted OpenBCI board that transmits the SSVEP 
signals to the Notebook computer via Bluetooth. The board samples the signals at 250 Hz via ADS1299 and 
PIC32MX250F128B micro-controller and powered with 6 V through four AA size batteries. The board size 
was 2.41” x 2.41” excluding battery pack. 

The graphical user interface (GUI) of modified OpenBCI program provided the real time SSVEP- 
EEG signals, visualized input, mode selection input, and status of 8 output channels. Real time SSVEP-EEG 
signals are the graphical representation of EEG activity at visual cortex. Information of visualized input 
gazed by the user is displayed instantly in the screen. Mode selection input allowing users to select the 
targeted operation as control penal or operating the computer mouse. The control panel is proposed to be of 8 
digital transistor-transistor logic (TTL) outputs to control any connected devices for home automation, bed 
adjustments, and alarms. User can toggle the status (OFF/ON) by gazing the respective LED. Status of these 
outputs were displayed on GUI in red and green colors with OFF/ON text. 


2.3. Data processing and control 

SSVEP signal obtained from visual cortex was processed in modified open source program of 
OpenBCI build on processing platform in Java. The signal was first filtered using 5-50 Hz bandpass filter and 
50 Hz notch filter. Classification of different SSVEP commands corresponding to the visualized LEDs were 
carried out using built-in feature of fast fourier transform (FFT). The amplitude of frequencies was directly 
obtained from the filtered data ready for FFT plot. A command for particular visual stimuli was considered 
TRUE if amplitude at corresponding frequency is highest. A TRUE command toggles the state of the 
designated feature of mode selection from ‘control panel’ to ‘computer mouse’ or vice versa for mode 
selection stimuli. The TRUE commands then trigger the cursor to move towards corresponding direction or 
to perform the click in mouse operation. Similarly, it toggles the output state of corresponding control panel 
output pin when control panel operation was selected. 


3. EXPERIMENTAL RESULTS AND DISCUSSION 
The experiments were conducted in a softly-lit room in three different sessions in which 
participants were seated in a comfortable chair wearing electrodes in front of experimental setup. 


3.1. Experimental session 1 

In first session, participants were asked to gaze each flickering LED for 3 seconds with 3 seconds 
interval in response of manual audio cue to confirm the stimuli detection by the experimental setup. The trails 
were repeated 5 times and SSVEP responses were recorded which were then used to compute the 
corresponding amplitude level of each visual stimulus obtained in each trial. Corresponding amplitudes 
values with minimum, maximum, mean, and standard deviation are described in Figure 3. The participants 
were also filled a questioner to rate the comfort level of gazing each flickering LEDs on a scale of 10 that are 
described in Figure 4. 
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Figure 3. Max, min, mean, and standard deviation in SSVEP Amplitudes (uV) at different flickering 
frequencies 
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Figure 4. Level of comfort in gazing LEDs at different frequencies-least comfort (0)/most comfort (10) 


The SSVEP amplitudes correspond to each frequency obtained through this low cost setup are above 
5.5 uV and reached up to 9.6 uV that are quite enough to detect the corresponding SSVEP frequencies. The 
results indicating higher amplitudes at frequencies from 21 Hz to 31 Hz, therefore these stimuli are used for 
more critical inputs like mode selection and control panel operation. Experiments also indicating the 
tendency of decreasing amplitudes at frequencies lower than 14 Hz and higher than 37 Hz. This is limiting 
the possibility of addition of more stimuli beyond 8 Hz and 40 Hz. The participants reported very less 
comfort level at lower frequencies and good comfort at higher frequencies. This is due the eye discomfort by 
continuously gazing low frequency stimuli which is less at higher frequencies. Lindquist et al. [32] reported 
that short term disclosure of uncomfortable chromatic stimuli is not creating harmful effect on visual task 
performance except creating a sharp neural response across cortex. However, to provide comfort to the users, 
continuous exposure to low frequency stimuli should be avoided. For this reason, higher frequency stimuli 
are used in this system to trigger mouse control operation that requires continuous gazing of desired stimuli. 
Further research may consider the use of other visual stimuli to address the eye discomfort like Mouli and 
Palaniappan [33] presented in a work aiming to develop a hybrid BCI platform for stimuli generation to 
evoke SSVEP and P300 to minimize fatigue with improved classification performance. 


3.2. Experimental session 2 

Second session was designed to test the accuracy of control panel operation in which participants 
were first selected the control panel by gazing the mode selection LED. Then they asked to toggle each 
output by gazing the corresponding stimuli for 2 seconds in response of audio cue with different time 
intervals of 1 to 5 seconds. Each output was tested 10 times and the output states on control panel were 
recorded to perform offline analysis for efficiency/accuracy of trials on the basis of success and failure of 
triggering the output pins each time. Average success rates of each operation are given in Table 1. 
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Table 1. Success rate out of 10 attempts in mode selection and control panel operation 
Operation Success Rate 


Mode Selection 70% 
Switch-1 60% 
Switch-2 63% 
Switch-3 60% 
Switch-4 61% 
Switch-5 60% 
Switch-6 63% 
Switch-7 61% 
Switch-8 61% 


Experimental results of this trial confirm the efficiency of control panel operation up to 60% or at 
higher success rate. Since this was a training less trial, subjects may exhibit more efficient success rate after 
performing repeated trials or after use in day to day tasks. However, further research may be conducted to 
improve the efficiency by incorporating some new features in processing techniques and stimuli design. 


3.3. Experimental session 3 

Third session examined the mouse control operation in which participants were asked to visualize 
the corresponding LEDs to move the cursor from center of the screen C to each corner of the screen referred 
to as E, F, G, and H in clockwise manner. The time to move the cursor from one point to next by each 
participant were recorded to calculate the time consumed to move the cursor step by step. Individual and 
average time for each operation are given in Table 2 and Figure 5. 


Table 2. Cursor movement timings in seconds for each step 


Step Pl P2 P3 P4 P5 P6 P7 P8 Average 
CtoE 4 4 3 5 4 4 5 3 4.0 
E to F 6 6 5 7 6 7 6 5 6.0 
F to G 6 5 5 7 6 7 8 7 6.4 
G to H 7 7 6 8 7 8 8 6 7.1 
H to E 8 7 8 9 9 9 9 8 8.4 
Overall 31 29 27 36 32 35 36 29 31.9 


Cursor Movement Timings in Seconds 


EH to fk MGtoH OBFtoG Eto Ft BCtotk 





Figure 5. Cursor movement time for each step (in seconds) 


The time to move cursor from center to the first corner of the screen is less due to the less distance. 
Time consumed to move cursor from E to F, F to G, G to H, and H to E increased respectively due to the use 
of successive higher frequency stimuli that are of relatively less SSVEP amplitudes and exhibiting more false 
negative triggers as compared to former corresponding stimuli. At the end of experiments, participants were 
asked to fill the questionnaire to rate the overall experience in selection of mode, mouse control in each 
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direction and operating the output of control panel. They were also asked to rate the overall comfort level 


during the trials. The response of the participants is given in Table 3. 


Table 3. Level of easiness in operations-least comfort (0)/most comfort (10) 
Operation P1 P2 P3 P4 P5 P6 P7 P8 _ Average 


Mode Selection 10 9 10 9 9 9 8 10 9.3 
Control Panel 7 8 7 7 7 8 7 8 7.4 
Mouse Pointer 6 5 5 5 5 5 5 6 5.3 
Overall Experience 8 8 7 8 7 7 8 7 7.5 


Mode selection operation was reported most easy as compared to control panel and mouse pointer 
control as the mode selection is not required to be triggered continuously. Mouse pointer control was 
reported comparatively less easy due to the eye discomfort in this operation due to continuous gazing of 
stimuli. Overall cost of the setup is quite less as compared to other commercially available systems as 
described in Table 4. 


Table 4. Comparison of proposed setup with other commercially available systems. 
Open Source HW Price of Components Usability*_ Efficiency 


g.tec g. USBamp No Very high Yes Very High 
Emotive Epoc No Medium No Medium 
Neuroscan SybAmps No High Yes High 
Neuroscan NuAmps No High Yes High 
Neurosky Mindware No Low No High 
Olimex EEG-SMT Yes Low No Medium 
Proposed System Yes Low Yes Medium 


*Wear-ability and comfort for locked-in patients 


First five systems are not based on open source hardware and software and have relatively high 
costs. Olimex EEG-SMT is an open source hardware which is used by Acampora et al. in their BCI 
experiment [34] but it requires dry electrode consisting of 12 pointy pinches to be placed at the back of the 
scalp which is limiting its use for locked in patients as they are not able to use the system at sitting position. 
The proposed system is based on open source hardware with low components cost and providing medium 
level of efficiency with wear-ability and comfort for locked-in patients. To the best of our knowledge, the 
proposed system is a cost effective solution with features of comfort and wear-ability particularly for the 
users of low and middle-income countries. 


4. CONCLUSION 

This paper presents a low-cost general purpose HCI system that can be used to operate computers 
and a control panel through brain signals. The proposed system comprises of a low cost bio-signal amplifier 
interfaced with eight electrodes to acquire SSVEP-EEG signals through gazing LEDs flickering at different 
frequencies. These inputs were used to operate a control panel of eight discrete outputs and to operate a 
computer mouse. The experimental results confirm that this low cost implementation can be beneficial for 
completely locked-in patients and will enable them to use computer and IT devices in a cost efficient manner 
with acceptable efficiency and accuracy up to 60% or higher. Further research may target improvements in 
the efficiency of the system incorporating some new processing techniques and stimuli design in low cost 
manner to supplement this proposed system. 
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